Individuals, institutions, artists, video game makers, movie makers, researchers, and others may seek to generate synthetic videos and/or alter real videos by inserting synthetic objects that have synthetic motion. Synthetic videos include tutorials, motion pictures, video games, public displays (e.g., airport safety videos), training videos, and other applications. Synthetic videos may protect confidentiality, portray situations too dangerous to film in the real world, and/or portray impossible situations (e.g., epic fantasy scenes). Synthetic videos can reduce video production costs or otherwise meet video needs when live-videos are inadequate. However, tools generating synthetic realistic motion may not exist in some systems, and conventional systems of generating synthetic motion are limited resulting in high production costs and/or poor quality videos.
For example, conventional approaches to generating synthetic videos include using models to create a sequence of images that comprise a video. Some synthetic videos may be created by filming live action and mapping synthetic features onto the live action (e.g., altering physical appearance from a human to a non-human creature, or mapping synthetic objects onto real-world moving props). These methods of producing synthetic videos require tightly controlled filming conditions (e.g., using particular filming conditions or involving specialized equipment) and involve many people, leading to high production costs and long production times.
To address these problems, fully synthetic (computer generated) videos may be created. However, fully synthetic videos often suffer from unrealistic motion, including object distortion, or unnaturally abrupt (jerky) motion. For example, fully synthetic videos that depict a person walking may result in unrealistic arm, leg, torso, or head movements, or may result in distortions of facial features. In some systems, unrealistic motion arises because models used to generate motion are not based on underlying properties of motion and/or are not based on real videos.
Further, conventional synthetic videos often display problems associated with integrating synthetic objects into the video setting. For example, the shadows of a synthetic object in a video (e.g., synthetic shadows) may appear in an unrealistic position, with unrealistic shapes, and/or with unrealistic transparency. Synthetic objects may be unrealistically layered in front of or behind real objects or other synthetic objects. Synthetic objects may have unnatural or unwanted sizes or poses relative to background objects.
Therefore, in view of the shortcomings and problems with conventional approaches to synthetic video, there is a need for improved, unconventional systems that are low-cost, rapid systems to generate synthetic videos that portray realistic motion and address problems associated with integrating synthetic objects into a video setting.